Strabismus is the misalignment of the visual axis between the two eyes, and is the chief cause of amblyopia—reduced visual acuity in one eye which cannot be improved by glasses. Amblyopia is responsible for loss of vision in one eye in children and young adults more than all other causes combined in the Western world. It was found that 2%-4% of children develop strabismus at an early stage. Therefore, early detection of strabismus and amblyopia is essential if it is to be treated effectively. The efficiency of known treatment techniques reduces with age up until 7 or 8 years, after which the loss of vision is permanent. Numerous medical bodies around the world, such as the American Academy of Paediatricians, recommend vision screening no later than the age of 4, if not earlier. Nevertheless, children younger than 5 are difficult to screen, and less than 1 in 3 from this age group in the US undergo vision tests. Also, there are various other conditions which may cause amblyopia, such conditions include anisometropia which is a difference in refractive power between the two eye lenses, and opacities being a condition of cataract.
The conventional tests for strabismus detection and diagnosis are typically manual tests, and have to be performed by professionals. These manual tests include the Hirschberg Test and the Cover Test and are best performed by ophthalmologists and orthoptists. The Hirschberg test involves shining a light into patient's eyes and visually inspecting the eyes for symmetry in the reflection of light (the Purkinje image) from the corneal surface. The Cover Test involves covering one eye at a time and observing eye movement while the patient fixates on a specific target. Strabismus and other conditions may also be inspected via the Bruckner Test, which is also a manual (visual) test. These tests are difficult to perform on non-cooperating patients, such as infants and young children. Front line medical staff, such as general practitioners and paediatricians, typically screen patients for strabismus, but as they are not skilled experts for these manual tests, this leads to both under-referrals and over-referrals.
Attempts have been made to develop techniques facilitating determination of various eye diseases/abnormalities. For example, U.S. Pat. No. 7,854,509 describes an article and method for screening vision that does not require verbal input from a test person or the test person to maintain a fixed position during the screening. The article includes an image capture device, at least one test light, a processing unit that includes an interpretive program, and a display. The method comprises capturing an image of a person's eyes, verifying the image, studying the image, evaluating the image, and displaying the results. Conveniently, the article and method require no special training by an operator.
General Description
There is a need in the art for a novel technique for simple, automatic detection of various eye conditions, such as strabismus, anisometropia, anisocoria, as well as retinoblastoma and cataract.
The present invention provides a device according to claim 1. Further optional embodiments and aspects of the invention are described in claim 2 et seq.
In accordance with one aspect of the invention there is provided an apparatus and method for fully automatic detection of eye conditions associated with abnormalities in a subject's eyes. The technique can be used for identifying eye conditions in children or infants or other subjects which may be less cooperative using an inspection procedure. The inspection procedure may be simple and quick and does not require a skilled professional operator. Moreover, the inspection may utilize data indicative of a single image of the subject's eyes, where the image may be acquired by a digital camera typically installed in most portable electronic devices, such as mobile phone devices, smartphones, as well as portable computer, PDA, etc. Also, a device of the present invention may be installed in a digital camera or ophthalmoscope, specifically designed for implementing the invention.
The conventional tests for diagnosing strabismus, e.g. the Hirschberg test, are manual tests which require a skilled professional and are limited to detection of medium to large amplitude strabismus. When testing for eye conditions in children/infants, who are typically less cooperative, the test is even less accurate or reliable. An automatic (computerized) system for identifying eye conditions can provide more accurate diagnostics, and can provide at least preliminary results in a very short time, while requiring less patient cooperation.
The present invention provides for a novel approach for processing and analyzing image data, which enables the technique or processor utility of the present invention to be embedded in any electronic device, being a personal computer, laptop, or any hand held computing device such as a smartphone, tablet, digital camera, or ophthalmoscope. The electronic device may utilize a camera unit installed therein to acquire one or more images of a region of interest and provide image data, or it may be connectable (via wires or wireless signal transmission) to an external camera unit or storage device to receive image data therefrom, thus operating in either on-line (real time) or off-line inspection mode. The electronic device configured for carrying out the present invention thus comprises a processor utility which is configured as a software application preprogrammed to receive and analyze image data, and may also include or be connectable to an output utility (e.g. screen) for providing analysis results and other instructions to an operator.
In the non limiting examples described below, the device of the invention is referred to as a computing device which includes a camera unit integral with the device. It should, however, be understood that the present invention is not limited to such an integral device configuration, and may be implemented as a chip or software/hardware utility that may be embedded in any electronic or computing device for receiving and processing image data including but not limited to cameras, smartphones, mobile phones, ophthalmoscopes.
In the simplest example, where the electronic device of the invention is integral with a camera unit (e.g., a hand held computing device including a camera unit), a user operates the camera unit to acquire one or more images of a subject's face utilizing flash light or other light source. Image data is received by the processor utility which is preprogrammed according to the invention for analyzing this data, identifying abnormal eye conditions of the subject, and generating output data indicative thereof. To this end, the processor utility performs a so-called pre-processing to identify existence and location of a region of interest (i.e. the left and/or right eye region(s) of a subject) within the image in order to determine whether said image is a proper one to be used for further processing or another image is needed. The proper zone/region is that including desired eye features for detection of one or more abnormalities in the subject's eye(s). Such eye regions may for example include a pupil and a bright zone associated with reflection of a light source (light reflection from the eye surface is commonly known as a Purkinje image) for at least one of the left and right eyes of a subject.
According to one broad aspect of the invention, there is provided a portable electronic device comprising a processor utility configured and operable for processing image data, determining one or more conditions indicative of an abnormality in a subject's eye, and generating corresponding output data, said processor utility comprising: a first processor configured and operable for selecting in the image data an image comprising a region of interest including predetermined eye features of at least one of two eyes of the subject; an eye feature detection utility configured and operable for applying an image processing algorithm to the region of interest in the selected image for determining one or more parameters of the eye features from a set of predetermined parameters; and an eye condition identifier utility configured and operable for processing and analyzing said one or more parameters and generating data indicative of a condition of the subject's eyes.
The set of predetermined parameters of the eye features may include at least one of geometrical, positioning and color related parameters of the eye features. Such parameter(s) of the eye features may comprise one or more of the following: location of the pupil for at least one of the eyes; location of the Purkinje image for at least one of the eyes; physical dimension of at least one of the pupils; physical dimension of at least one Purkinje image, average color of the pupils of both eyes, and average color of the Purkinje images of both eyes.
The region of interest is a part of the selected image including at least one of left and right eye regions, and preferably both of these eye regions. The region of interest in the selected image may also be that comprising at least one feature corresponding to predetermined imaging conditions. Such a feature corresponding to the predetermined imaging conditions may comprise a Purkinje image.
The device preferably comprises a memory utility for storing predetermined reference data. The latter includes first reference data indicative of pixel values corresponding to face features, and second reference data indicative of pixel values corresponding to eye features.
In some embodiments of the invention, the first processor module is configured and operable for carrying out the image selection by processing an image using the first reference data to obtain rough data about the region of interest in said image based on classifying pixels in the image according to the first reference data and grouping pixels into different categories relating to the region of interest.
In some embodiments of the invention, the eye feature detection module is configured and operable for processing the region of interest using the second reference data to obtain rough value of said one or more of the geometrical and/or positioning and/or color related parameters of the eye features by classifying pixels in the region of interest according to said second reference data and counting pixels relating to at least one eye feature. The eye feature detection module may use the rough values of one or more of such parameters for re-classifying the pixels related to said at least one feature, and weighting values of said re-classified pixels.
The eye features to be determined comprise at least one of pupil and Purkinje image in at least one of left and right eye regions. As indicated above, the parameters of the eye features may comprise location of the pupil for at least one of the eyes; location of the Purkinje image for at least one of the eyes; physical dimension of at least one of the pupils; physical dimension of at least one Purkinje image; and/or average color of the pupil and/or Purkinje images of one or both eyes.
In some embodiments of the invention, the eye condition identifier module is configured and operable to determine a relation between the parameters of at least two eye features, wherein said at least two eye features relate to the same eye region or to the different eyes' regions.
In some embodiments of the invention, the eye condition identifier module determines the relation between relative positions of the center of the Purkinje image and the center of the pupil in the right eye to those of the left eye, and identifies a deviation of said relation from a predetermined value. Such deviation is indicative of a degree of asymmetry corresponding to strabismus condition. The predetermined value may be a certain function of a difference between the relative positions in the left and right eye regions and a distance between the left and right pupils.
In some embodiments of the invention, the eye condition identifier module determines said relation in the form of a vector distance between the center of the pupil and the center of the Purkinje image in at least one of the left and right eye regions in the image, and identifies a deviation of said relation from a predetermined value. This deviation is indicative of strabismus condition of said at least one eye.
In some embodiments of the invention, the eye condition identifier module determines a relation between radii of the Purkinje image of the left and right eye regions, and identifies a deviation of said relation from a predetermined value being indicative of anisometropia condition.
In some embodiments of the invention, the eye condition identifier module determines a relation between average colors of pixels in the Purkinje image and/or pupil of the left and right eye regions, and identifies a deviation of said relation from a predetermined value. For example, such deviation from a predetermined value (threshold value) is indicative of anisometropia condition.
In some embodiments of the invention, the eye condition identifier module determines a relation between average colors of pixels in the pupil of the left and right eye regions, and identifies a deviation of said relation from a predetermined value being indicative of the presence of retinoblastoma or of a cataract in the eye.
In some embodiments of the invention, the eye condition identifier module is configured and operable to determine a relation between radii of pupils in the left and right eye regions, and to identify a deviation of said relation from a predetermined value being indicative of anisocoria condition.
The device of the present invention may be configured as or incorporated in a phone device (e.g. smartphone), a PDA device, a personal computer, a digital camera, tablet, laptop, or a specially designed ophthalmoscope.
The first processor module may be configured and operable for processing the image data and upon identifying image data pieces in the selected image corresponding to the subject's eyes, extracting said image data pieces to be processed by the eye feature detection module.
The processor utility may be configured and operable to operate the imager (internal or external) for generating the image data including a stream of images to be processed by the first processor module to select the proper image. In some embodiments, the processor utility selectively operates to generate a request data for additional image(s).
In a specific embodiment of the invention, the processing of the image data comprises the following: locating right and left eye regions in the selected image; locating a zone in the eye region corresponding to a Purkinje image, for right and left eyes of the subject; determining a dimension of the Purkinje image for the right and left eye regions; determining a relation between the dimension of said right and left Purkinje images, analyzing data indicative of said relation and generating data indicative of a anisometropia condition of the subject.
In a specific embodiment of the invention, the processing of the image data comprises the following: locating right and left eye regions in the selected image; locating zones corresponding to pupils of the right and left eye regions; locating a zone in the eye region corresponding to a Purkinje image, for right and left eyes of the subject; determining a relative location of the Purkinje image with respect to the pupil, for the right and left eye regions; and analyzing said relative location, and generating data indicative of a strabismus condition of the subject.
According to another broad aspect of the invention, there is provided a hand-held electronic device comprising: a frame grabber for acquiring images of the surroundings and generating image data including at least one image of at least one eye of a subject, a light source for illuminating said surroundings at least during a time of image acquisition, and a processor utility connected to said imager and configured and operable for processing said at least one image, determining one or more conditions indicative of an abnormality in the subject's eye(s), and generating corresponding output data.
According to yet further aspect of the invention, there is provided a method for non-invasively screening subjects for certain one or more eye conditions using a hand-held device comprising a camera unit, a processor utility, a memory utility and a user interface utility. The method comprises:
a. operating the camera unit for providing image data indicative of one or more images taken by the camera unit at a certain distance from a subject, said distance ranging from a few centimeters to a few meters,
b. processing said image data for validating presence therein of at least one image of one or both eyes of the subject;
c. processing said at least one image of one or both eyes of the subject and determining at least one of position, dimension and color related parameter of at least one eye feature including at least one of pupil and Purkinje images, and analyzing said at least one of the position, dimension and color parameters of at least one eye feature with respect to predetermined threshold data; and
d. generating data indicative of existence or absence of one or more of the eye conditions.
According to yet another broad aspect of the invention, there is provided a control unit for embedding in an electronic device, said control unit comprising a connection utility for connecting to an imager, and a processor utility configured and operable for processing image data received from the imager and generating output data indicative of one or more abnormalities in a subject's eyes being imaged, said processing comprising determination of at least one of geometrical, positioning and color related parameters of eye features in at least one of right and left eyes of a subject.
According to yet further broad aspect of the invention, there is provided a computer program product embedded on a non-transitory computer-accessible medium and comprising a computer readable program code configured and operable for processing image data corresponding to subject's eyes and generating output data indicative of one or more abnormalities in the subject's eyes, said processing comprising determination of at least one of geometrical, positioning and color related parameters of eye features in at least one of right and left eyes of a subject.
The computer program product is capable selecting in the image data an image comprising a region of interest including predetermined eye features of at least one of two eyes of the subject; applying an image processing algorithm to the region of interest in the selected image for determining one or more parameters of the eye features from a set of predetermined parameters; and processing and analyzing said one or more parameters and generating data indicative of a condition of the subject's eyes. To this end, the computer program product accesses certain first and second reference data indicative of pixel values corresponding to respectively face features and eye features. Such reference data may be stored in an external storage utility to which the program has access or may actually be a part of the program itself. The program uses the first reference data for processing the image data to obtain rough data about the region of interest based on classifying pixels in the image according to said first reference data and grouping pixels of different categories relating to the region of interest, thereby selecting in the image data a proper image comprising a region of interest. Then, the second reference data is used for processing the selected proper image to obtain rough value of said at least one parameter of the eye features by classifying pixels in the region of interest according to said second reference data and counting pixels relating to at least one eye feature.
According to yet another aspect of the invention, there is provided an electronic device comprising a processor utility configured and operable for processing image data, determining one or more conditions indicative of an abnormality in a subject's eye, and generating corresponding output data, said processor utility comprising:
a first processor configured and operable for selecting in the image data an image comprising a region of interest including predetermined eye features of at least one of two eyes of the subject, said selecting comprising processing an image using a first reference data indicative of pixel values corresponding to face features to obtain rough data about the region of interest in said image based on classifying pixels in the image according to said first reference data and grouping pixels of different categories relating to the region of interest;
an eye feature detection utility configured and operable for applying an image processing algorithm to the region of interest in the selected image for determining one or more of geometrical and positioning parameters of the eye features, said applying of the image processing algorithm comprising using second reference data indicative of pixel values corresponding to eye features to obtain a value of said one or more of geometrical and positioning parameters of the eye features by classifying pixels in the region of interest according to said second reference data and counting pixels relating to at least one eye feature; and
an eye condition identifier utility configured and operable for processing and analyzing said one or more geometrical and positioning parameters and generating data indicative of a condition of the subject's eyes.